Rate computations of particular use in scheduling activities or items such as the sending of packets

ABSTRACT

Rate computations are performed such as for use in scheduling activities, such as, but not limited to packets, processes, traffic flow, etc. One implementation identifies an approximated inverse rate, a fix-up adjustment value, and a quantum. An activity measurement value is maintained based on a measure of activity, and a rate control value is maintained based on the measure of activity and the approximated inverse rate. The fix-up adjustment value is applied once each quantum to the rate control value to maintain rate accuracy of the activity. In one implementation, the control value is a scheduling value used for determining when to perform a next part of the activity (e.g., send one or more packets). Scheduling rates are efficiently and compactly stored in an inverse form, which may have advantages in terms of rate granularity, accuracy, and the ability to deliver service smoothly.

TECHNICAL FIELD

One embodiment of the invention relates to communications and computersystems, especially networked routers, packet switching systems, andother devices; and more particularly, one embodiment relates to ratecomputations for metering the rate at which things occur, which may beof particular use in scheduling packets.

BACKGROUND

The communications industry is rapidly changing to adjust to emergingtechnologies and ever increasing customer demand. This customer demandfor new applications and increased performance of existing applicationsis driving communications network and system providers to employnetworks and systems having greater speed and capacity (e.g., greaterbandwidth). In trying to achieve these goals, a common approach taken bymany communications providers is to use packet switching technology.Increasingly, public and private communications networks are being builtand expanded using various packet technologies, such as InternetProtocol (IP).

A network device, such as a switch or router, typically receives,processes, and forwards or discards a packet. For example, an enqueuingcomponent of such a device receives a stream of various sized packetswhich are accumulated in an input buffer. Each packet is analyzed, andan appropriate amount of memory space is allocated to store the packet.The packet is stored in memory, while certain attributes (e.g.,destination information and other information typically derived from apacket header or other source) are maintained in separate memory. Oncethe entire packet is written into memory, the packet becomes eligiblefor processing, and an indicator of the packet is typically placed in anappropriate destination queue for being serviced according to somescheduling methodology.

Certain packet types and classifications of packet traffic must be sentat certain rates for reasons such as the nature of the traffic or a ratelevel guaranteed by a service provider, wherein the term “rate” as usedherein typically refers to real-time rate and/or the resultant effect ofany weighted service policy, such as, but not limited to virtual timeweights, tokens, credits, events, etc. Thus, a scheduling system mustdeliver service at a specified rate to a queue containing packets ofvarying sizes. Furthermore, the rate delivery system must be able tosupport both real-time rate delivery (e.g., a fixed number of bytes persecond), and virtual time rate delivery (e.g., a weighted fraction ofthe total available bandwidth). Rates must be encoded and stored, andcomputed and tracked in a fashion that is easily interpreted by hardwareand/or software. For calendar-based schedulers, this rate is normallyencoded as a quantum (i.e., a number of bytes) served in an interval(i.e., a number of calendar slots).

Quantum/interval encoding has several problems, including variable rateaccuracy and a burstiness property. At the fast end of the range, theaccuracy is n bytes (i.e., the quantum) in one interval, as the rate canonly be changed by varying the number of bytes sent, and hence theaccuracy is related to the maximum transmission unit (MTU). Thus, a tenthousand byte MTU would offer one part in ten thousand, but a fifteenhundred byte MTU (e.g., that used in Ethernet) would only offer one partin fifteen hundred. The quantum/interval scheme does not deliver ratessmoothly. For example, with a ten thousand byte quantum, a queue sendingforty byte packets might need to burst two hundred fifty packets beforeit was rescheduled. If its interval was greater than one, it would bepreferable to reschedule it in intermediate steps as it sent eachpacket, rather than after all two hundred fifty packets are sent.

Most known systems have used some variant of the quantum/intervalapproach, where on each service of some fixed quantum of bytes, thecalendar is advanced by a certain interval. Generally, these systemshave either used large quantums more than the size of an MTU, or theyhave had to employ other techniques to deal with quantums that are lessthan an MTU. Larger quantums avoid complexities in the implementation,but the trade-off is much more burstiness.

Some systems have mitigated the burstiness problem by using a quantumthat is smaller than an MTU. However, because packets then can be muchlarger than one quantum, a division operation (i.e., size/quantum) isrequired to compute the number of calendar slots to be moved. While thisimproves smoothness of rate delivery, it does so only at a trade-off inaccuracy as the use of smaller quantums to deliver rates exacerbates thevariable rate accuracy issue. Also, division is typically a veryexpensive operation. If a hardware divide capability is not available(as on many embedded software platforms), either the quantum must berestricted to a power of two which results in rate granularity problems,or the division must be done iteratively in which case rate computationdoes not operate in a fixed time). Moreover, using a hardwareimplemented divide operation could also introduce issues with“drift”/round-off errors which cause some of the desired rate to belost. Accordingly, prior systems can offer granularity, but trade offsmoothness against accuracy. Further, the need to createquantum/interval pairs to encode rates and the constraints on thecreation of those pairs, can make it difficult to configure suchsystems. In particular, systems which use a power-of-two-only quantummay require iterative procedures to define sets of rates that best meettheir individual criteria, and their relationship to each other.

SUMMARY

Disclosed are, inter alia, methods, apparatus, data structures,computer-readable medium, mechanisms, and means for performing ratecomputations, which may be used in most anything that meters the flow atwhich things happen. As such, these rate computations may beparticularly useful in scheduling activities or items, such as, but notlimited to packets (especially a series of related packets), processes,threads, traffic flow of any nature, etc.

One embodiment identifies an approximated inverse rate, a fix-upadjustment value, and a quantum. An activity measurement value ismaintained based on a measure of activity, and a rate control value ismaintained based on the measure of activity and the approximated inverserate. The fix-up adjustment value is applied once each quantum to therate control value to maintain rate accuracy of the activity. In oneembodiment, the control value is a scheduling value used for determiningthe relative ordering or timing for performing a next part of theactivity (e.g., send one or more packets). Scheduling rates areefficiently and compactly stored in an inverse form, which may haveadvantages in terms of rate granularity, accuracy, and the ability todeliver service smoothly. In one embodiment, applying the fix-upadjustment value once each quantum to the rate control value includesdithering the rate control value to either round-up or not to round-upthe rate control value based on a random number

One embodiment associates with a scheduling flow, such as a series ofpackets, a current slot, a scheduling item corresponding to a packet,and an approximated inverse rate, a fix-up adjustment value, and aquantum value. A last adjusted slot corresponding to the scheduling itemis identified. A bytes sent value is adjusted based on a number of bytesof the packet to identify a new bytes sent value. In response toidentifying that the bytes sent value is greater than or equal to aquantum value corresponding to the scheduling item: (a) a new lastadjusted slot for the scheduling item is identified, which typicallyincludes summing a product of the approximated inverse rate and thequantum value, the fix-up adjustment value, and the last adjusted slot;and (b) a next slot for the scheduling item is determined, whichtypically includes adding the product of the approximated inverse rateand the new bytes sent value to the new last adjusted slot.

In one embodiment, identifying the last adjusted slot for the schedulingitem includes subtracting the product of the approximated inverse rateand the bytes sent value from the current slot. In one embodiment, thefix-up adjustment value is determined based on the error induced by theapproximated inverse rate during a quantum corresponding to thescheduling item. In one embodiment, in response to identifying that thebytes sent value is less than a quantum value corresponding to thescheduling item, the next slot is determined, which typically includesadding the product of the approximated inverse rate and the new bytessent value to the last adjusted slot. In one embodiment, identifying thenew last adjusted slot for the scheduling item includes dithering thenew last adjusted slot to either round-up or not to round-up the newlast adjusted slot based on a random number.

BRIEF DESCRIPTION OF THE DRAWINGS

The appended claims set forth the features of the invention withparticularity. The invention, together with its advantages, may be bestunderstood from the following detailed description taken in conjunctionwith the accompanying drawings of which:

FIG. 1A is a flow diagram illustrating a process for performing ratecomputations used in one embodiment;

FIG. 1B is a block diagram illustrating stored values used in performingrate computations in one embodiment;

FIG. 1C illustrates the computation of an adjusted slot as performed inone embodiment;

FIG. 1D illustrates pseudo-code of a process for performing ratecomputations used in one embodiment;

FIG. 2 is a block diagram illustrating stored values used in performingrate computations in one embodiment;

FIG. 3A is a flow diagram illustrating a process for determining anapproximated inverse rate, a fix-up adjustment value, and a quantumvalue used in one embodiment;

FIG. 3B illustrates an exemplary resultant set of approximated inverserates, fix-up adjustment values, and a quantum values for various targetrates used in conjunction with one embodiment;

FIG. 4A is a block diagram of a system to perform the rate computationsand/or determination of approximated inverse rates, fix-up adjustmentvalues, and a quantum values used in one embodiment; and

FIG. 4B is a block diagram of a system to perform the rate computationsused in one embodiment.

DETAILED DESCRIPTION

Disclosed are, inter alia, methods, apparatus, data structures,computer-readable medium, mechanisms, and means for performing ratecomputations, which may be used in most anything that meters the flow atwhich things happen. As such, these rate computations may beparticularly useful in scheduling activities or items, such as, but notlimited to packets (especially a series of related packets), processes,threads, traffic flow of any nature, etc.

Embodiments described herein include various elements and limitations,with no one element or limitation contemplated as being a criticalelement or limitation. Each of the claims individually recites an aspectof the invention in its entirety. Moreover, some embodiments describedmay include, but are not limited to, inter alia, systems, networks,integrated circuit chips, embedded processors, ASICs, methods, andcomputer-readable medium containing instructions. One or multiplesystems, devices, components, etc. may comprise one or more embodiments,which may include some elements or limitations of a claim beingperformed by the same or different systems, devices, components, etc.The embodiments described hereinafter embody various aspects andconfigurations within the scope and spirit of the invention, with thefigures illustrating exemplary and non-limiting configurations.

As used herein, the term “packet” refers to packets of all types or anyother units of information or data, including, but not limited to, fixedlength cells and variable length packets, each of which may or may notbe divisible into smaller packets or cells. The term “packet” as usedherein also refers to both the packet itself or a packet indication,such as, but not limited to all or part of a packet or packet header, adata structure value, pointer or index, or any other part or direct orindirect identification of a packet or information associated therewith.For example, often times a router operates on one or more fields of apacket, especially the header, so the body of the packet is often storedin a separate memory while the packet header is manipulated, and basedon the results of the processing of the packet (i.e., the packet headerin this example), the entire packet is forwarded or dropped, etc.Additionally, these packets may contain one or more types ofinformation, including, but not limited to, voice, data, video, andaudio information. The term “item” is used generically herein to referto a packet or any other unit or piece of information or data, a device,component, element, or any other entity. The phrases “processing apacket” and “packet processing” typically refer to performing some stepsor actions based on the packet contents (e.g., packet header or otherfields), and such steps or action may or may not include modifying,storing, dropping, and/or forwarding the packet and/or associated data.

The term “system” is used generically herein to describe any number ofcomponents, elements, sub-systems, devices, packet switch elements,packet switches, routers, networks, computer and/or communicationdevices or mechanisms, or combinations of components thereof. The term“computer” is used generically herein to describe any number ofcomputers, including, but not limited to personal computers, embeddedprocessing elements and systems, control logic, ASICs, chips,workstations, mainframes, etc. The term “processing element” is usedgenerically herein to describe any type of processing mechanism ordevice, such as a processor, ASIC, field programmable gate array,computer, etc. The term “device” is used generically herein to describeany type of mechanism, including a computer or system or componentthereof. The terms “task” and “process” are used generically herein todescribe any type of running program, including, but not limited to acomputer process, task, thread, executing application, operating system,user process, device driver, native code, machine or other language,etc., and can be interactive and/or non-interactive, executing locallyand/or remotely, executing in foreground and/or background, executing inthe user and/or operating system address spaces, a routine of a libraryand/or standalone application, and is not limited to any particularmemory partitioning technique. The steps, connections, and processing ofsignals and information illustrated in the figures, including, but notlimited to any block and flow diagrams and message sequence charts, maytypically be performed in the same or in a different serial or parallelordering and/or by different components and/or processes, threads, etc.,and/or over different connections and be combined with other functionsin other embodiments, unless this disables the embodiment or a sequenceis explicitly or implicitly required (e.g., for a sequence of read thevalue, process the value—the value must be obtained prior to processingit, although some of the associated processing may be performed priorto, concurrently with, and/or after the read operation). Furthermore,the term “identify” is used generically to describe any manner ormechanism for directly or indirectly ascertaining something, which mayinclude, but is not limited to receiving, retrieving from memory,determining, defining, calculating, generating, etc.

Moreover, the terms “network” and “communications mechanism” are usedgenerically herein to describe one or more networks, communicationsmediums or communications systems, including, but not limited to theInternet, private or public telephone, cellular, wireless, satellite,cable, local area, metropolitan area and/or wide area networks, a cable,electrical connection, bus, etc., and internal communications mechanismssuch as message passing, interprocess communications, shared memory,etc. The term “message” is used generically herein to describe a pieceof information which may or may not be, but is typically communicatedvia one or more communication mechanisms of any type.

The term “storage mechanism” includes any type of memory, storage deviceor other mechanism for maintaining instructions or data in any format.“Computer-readable medium” is an extensible term including any memory,storage device, storage mechanism, and other storage and signalingmechanisms including interfaces and devices such as network interfacecards and buffers therein, as well as any communications devices andsignals received and transmitted, and other current and evolvingtechnologies that a computerized system can interpret, receive, and/ortransmit. The term “memory” includes any random access memory (RAM),read only memory (ROM), flash memory, integrated circuits, and/or othermemory components or elements. The term “storage device” includes anysolid state storage media, disk drives, diskettes, networked services,tape drives, and other storage devices. Memories and storage devices maystore computer-executable instructions to be executed by a processingelement and/or control logic, and data which is manipulated by aprocessing element and/or control logic. The term “data structure” is anextensible term referring to any data element, variable, data structure,database, and/or one or more organizational schemes that can be appliedto data to facilitate interpreting the data or performing operations onit, such as, but not limited to memory locations or devices, sets,queues, trees, heaps, lists, linked lists, arrays, tables, pointers,etc. A data structure is typically maintained in a storage mechanism.The terms “pointer” and “link” are used generically herein to identifysome mechanism for referencing or identifying another element,component, or other entity, and these may include, but are not limitedto a reference to a memory or other storage mechanism or locationtherein, an index in a data structure, a value, etc. The term“associative memory” is an extensible term, and refers to all types ofknown or future developed associative memories, including, but notlimited to binary and ternary content addressable memories, hash tables,TRIE and other data structures, etc. Additionally, the term “associativememory unit” may include, but is not limited to one or more associativememory devices or parts thereof, including, but not limited to regions,segments, banks, pages, blocks, sets of entries, etc.

The term “one embodiment” is used herein to reference a particularembodiment, wherein each reference to “one embodiment” may refer to adifferent embodiment, and the use of the term repeatedly herein indescribing associated features, elements and/or limitations does notestablish a cumulative set of associated features, elements and/orlimitations that each and every embodiment must include, although anembodiment typically may include all these features, elements and/orlimitations. In addition, the phrase “means for xxx” typically includescomputer-readable medium containing computer-executable instructions forperforming xxx.

In addition, the terms “first,” “second,” etc. are typically used hereinto denote different units (e.g., a first element, a second element). Theuse of these terms herein does not necessarily connote an ordering suchas one unit or event occurring or coming before another, but ratherprovides a mechanism to distinguish between particular units.Additionally, the use of a singular tense of a noun is non-limiting,with its use typically including one or more of the particular thingrather than just one (e.g., the use of the word “memory” typicallyrefers to one or more memories without having to specify “memory ormemories,” or “one or more memories” or “at least one memory”, etc.).Moreover, the phrases “based on x” and “in response to x” are used toindicate a minimum set of items x from which something is derived orcaused, wherein “x” is extensible and does not necessarily describe acomplete list of items on which the operation is performed, etc.Additionally, the phrase “coupled to” is used to indicate some level ofdirect or indirect connection between two elements or devices, with thecoupling device or devices modifying or not modifying the coupled signalor communicated information. The term “subset” is used to indicate agroup of all or less than all of the elements of a set. The term“subtree” is used to indicate all or less than all of a tree. Moreover,the term “or” is used herein to identify a selection of one or more,including all, of the conjunctive items.

Disclosed are, inter alia, methods, apparatus, data structures,computer-readable medium, mechanisms, and means for performing ratecomputations, which may be used in most anything that meters the flow atwhich things happen. As such, these rate computations may beparticularly useful in scheduling activities or items, such as, but notlimited to packets (especially a series of related packets), processes,threads, traffic flow of any nature, etc.

One embodiment identifies an approximated inverse rate, a fix-upadjustment value, and a quantum. An activity measurement value ismaintained based on a measure of activity, and a rate control value ismaintained based on the measure of activity and the approximated inverserate. The fix-up adjustment value is applied once each quantum to therate control value to maintain rate accuracy of the activity. In oneembodiment, the control value is a scheduling value used for determiningthe relative ordering or timing for performing a next part of theactivity (e.g., send one or more packets). Scheduling rates areefficiently and compactly stored in an inverse form, which may haveadvantages in terms of rate granularity, accuracy, and the ability todeliver service smoothly. In one embodiment, applying the fix-upadjustment value once each quantum to the rate control value includesdithering the rate control value to either round-up or not to round-upthe rate control value based on a random number

One embodiment associates with a scheduling flow, such as a series ofpackets, a current slot, a scheduling item corresponding to a packet,and an approximated inverse rate, a fix-up adjustment value, and aquantum value. A last adjusted slot corresponding to the scheduling itemis identified. A bytes sent value is adjusted based on a number of bytesof the packet to identify a new bytes sent value. In response toidentifying that the bytes sent value is greater than or equal to aquantum value corresponding to the scheduling item: (a) a new lastadjusted slot for the scheduling item is identified, which typicallyincludes summing a product of the approximated inverse rate and thequantum value, the fix-up adjustment value, and the last adjusted slot;and (b) a next slot for the scheduling item is determined, whichtypically includes adding the product of the approximated inverse rateand the new bytes sent value to the new last adjusted slot.

In one embodiment, identifying the last adjusted slot for the schedulingitem includes subtracting the product of the approximated inverse rateand the bytes sent value from the current slot. In one embodiment, thefix-up adjustment value is determined based on the error induced by theapproximated inverse rate during a quantum corresponding to thescheduling item. In one embodiment, in response to identifying that thebytes sent value is less than a quantum value corresponding to thescheduling item, the next slot is determined, which typically includesadding the product of the approximated inverse rate and the new bytessent value to the last adjusted slot. In one embodiment, identifying thenew last adjusted slot for the scheduling item includes dithering thenew last adjusted slot to either round-up or not to round-up the newlast adjusted slot based on a random number.

One embodiment uses an approximate inverse rate, requiring fewer bitsthan a more accurate value of the inverse rate, and periodically adjuststhe next slot based on a predetermined fix-up adjustment value such thatthe next slot varies as it would if there was not the error induced bythe use of the approximation of the inverse rate. Thus, instead ofrelying on a large number of bits to represent the inverse rate and manyfractional bits for the current slot, one embodiment uses a byte countto adjust the slot value periodically (e.g., one or more times perquantum) to reduce or eliminate the induced error.

In one embodiment, an interval between adjusted starting slots forconsecutive quantums is defined as: interval=quantum*approximatedinverse rate+fix-up adjustment value. Accuracy is achieved by ensuringthat at least, but typically once for each quantum of bytes sent, theexact same number of desired slots is traversed. Because afloating-point value of the approximated inverse rate may not have theprecision to get to the proper point, the fix-up adjustment value isadded once per quantum.

FIG. 1A is a flow diagram illustrating a process for performing the ratecomputations used in one embodiment. Processing begins with processblock 100, and proceeds to process block 102, wherein the dynamicscheduling information is retrieved from a current scheduling slot andthe corresponding static rate information is retrieved, typically fromanother data structure.

FIG. 1B illustrates stored data items used in one embodiment. A set 120of relatively static items is retrieved from storage each schedulingcycle in one embodiment, with set 120 including an approximated inverserate 121, a fix-up adjustment value 122, and the corresponding quantum123. These items 121-123 are labeled “relatively static values” (alsoelsewhere just referenced as “static” values or information, etc.) asthey are typically defined once for the duration of the metering of theassociated packet stream, item series, etc. Thus, during scheduling, aread from memory operation is required without a write operation as dataitems 121-123 do not change each scheduling cycle. However, in oneembodiment, these items are periodically, occasionally or otherwiseadjusted (e.g., typically not every scheduling cycle).

In contrast, set 125 of dynamic scheduler specific items used in oneembodiment includes values of the bytes sent 126 and the currentscheduling slot 127, which are updated each scheduling cycle. Oneembodiment performs rate computations disclosed herein with a differentscheduling mechanism, and in which case, it is possible that set 125includes no values or a different set of dynamically stored values.

Returning to the processing of FIG. 1A, in process block 104, the lastadjusted starting slot corresponding to the current quantum ofinformation being set is identified. In one embodiment, an indication ofthe last adjusted starting slot is retrieved from memory. While in oneembodiment, this last adjusted starting slot is calculated bysubtracting the product of the approximated inverse rate and the numberof bytes previously sent from the current slot position. Next, inprocess block 106, one or more corresponding packets are sent and thevalue of the bytes sent variable is increased accordingly.

As determined in process block 108, if the number of bytes sent for thecurrent quantum equals or exceeds the value of the quantum, then inprocess block 110, a new last adjusted slot is determined based on thequantum, approximated inverse rate, and the fix-up adjustment value. Inone embodiment, the new last adjusted slot is determined by summing theproduct of the approximated inverse rate and the quantum, the fix-upadjustment value, and the last adjusted starting slot. Of course, thisvalue of the new adjusted starting slot is truncated or adjusted (e.g.,wrapped around) as needed to match the slot data structure. In processblock 112, the value of the number of bytes sent is reduced by the valueof the quantum such that the number of bytes sent is less than thequantum value. Of course, this may be accomplished by truncating thevalue or by other means (e.g., an overflow indication corresponding tothe quantum being equaled or exceeded with the current value of thenumber of bytes sent being the non-overflowed value, etc.).

In process block 114, the new slot is determined based on theapproximated inverse rate, the number of bytes sent since the lastadjustment of the starting slot, and the value of the last or adjustedstarting slot. In process block 116, the dynamic data (e.g., bytes sentand current slot) is stored back in the scheduling data structure at thenew slot position. Processing is complete as indicated by process block118.

FIG. 1C illustrates the computation of an adjusted slot as performed inone embodiment. One way to view the last adjusted slot is that it is theslot position corresponding to zero bytes of data being sent in thecurrent quantum. Last adjusted slot 150 is shown. During processing 151,a first packet is sent, with a resulting new slot 152 being determinedbased on the approximated inverse rate, the number of bytes sent, andthe last adjusted slot 150. During processing 153, a second packet issent, with a resulting new slot 154 being determined. Note, lastadjusted slot 150 is either retrieved from memory, or typicallycalculated to save storage space. Then, new slot 154 is determinedrelative to the calculated position of last adjusted slot 150.

Processing 155 corresponds to the sending of the nth packet in thecurrent quantum with the number of bytes sent exceeding the value of thequantum. The last adjusted slot is typically calculated, with a new lastadjusted slot 156 determined relative to last adjusted slot 150 bysumming the product of the approximated inverse rate and the quantum,the fix-up adjustment value, and the position of last adjusted slot 150.Slot 157 is then determined relative to new last adjusted slot 156 basedon the product of the approximated inverse rate and the number of bytesthat exceeded the last quantum.

FIG. 1D illustrates pseudo-code 180 of a process for performing ratecomputations used in one embodiment. Pseudo-code 180 describes a processsimilar to that illustrated in FIG. 1A. However, pseudo-code 180 furtherillustrates, inter alia, that dithering can be used when determining anext adjusted slot value to add additional accuracy.

In one embodiment, the quantum is chosen so that the interval betweenadjusted starting slots for consecutive quantums (i.e.,interval=quantum*approximated inverse rate+fix-up adjustment value)“rounds” to the desired integer value. In one embodiment, dithering isused on the fractional bits of the interval, if any, to take advantageof the extra precision to deliver more accuracy. The dithering approachtypically provides the same accuracy whether the interval is large (nearthe total number of slots) or small (near one). Dithering also improvesgranularity near the fast end of the range, because the range of ratesthat can be delivered is determined not just by the quantum, but by thecombination of the quantum and the number of bits in the interval.

Dithering can be used to further improve rate-delivery accuracy. Theinterval can be treated as a non-integer (having more precision), andthe fractional part of the interval can be used to adjust the slot byeither zero or plus one when re-scheduling, with the adjustment chosenas plus one a percentage of the time based on the fractional part of theinterval. Dithering only affects the interval computation, and it doesnot, for example, increase the number of slots. In one embodiment, thefractional value of the interval does not change over time. As arounding operation performed on a constant fractional value will alwaysresult in the same value, rounding of the fractional value will notproduce any increased accuracy. However, using dithering to round up ortruncate based on a corresponding probability will produce increasedaccuracy, without having to maintain the extra fractional bits.

For example, consider a schedule entry that has a reschedule interval of4.3. A random number (i.e., a true or approximated random number) can begenerated between zero and one. This random number can be compared tothe fractional value of the interval (i.e., 0.3), with the rescheduleinterval being rounded-up to the value of five if the random number isless than or equal to (or just less than) 0.3, else truncated to thevalue of four.

Alternatively, with eight bits of fraction in the interval, 0.3 can berepresented as 77/256 or 0.30078125. A free running, 8-bit randomgenerator can be compared to 77 on every reschedule. With a probabilityof approximately 0.3, the random number will be less than 77; when thishappens, the entry is rescheduled by a rounded-up interval of 5; therest of the time the entry is rescheduled by a rounded-down interval of4. This also delivers an average interval of approximately 4.3.

FIG. 2 is a block diagram of items 220 retrieved on which to performrate calculations in one embodiment. As shown, items 220 include anapproximated inverse rate 221, a fix-up adjustment value 222, thecorresponding quantum 223, and a scaling exponent 224. Of course, thenumber of bits used for each field and the scheme used to encode thevalue stored in each field varies between embodiments.

For example, in one embodiment, the value of approximated inverse ratefield 221 stored in an eight bit field ranges from 0-255, while thevalue used in computations is the stored value divided by two raised tothe width of the field (e.g., 8 bits yielding 256) scaled based onscaling exponent 224, such as value representing between 2^0 and 2^-15as shown in the examples of encoded rates illustrated in FIG. 3B. Whilein one embodiment, the value of approximated inverse rate field 221stored in an eight bit field ranges from 128-255, while the value usedin computations is the stored value divided by two raised to the widthof the field (e.g., 8 bits yielding 256) scaled by some value, such asbetween 2^0 and 2^-15 as shown in the examples of encoded ratesillustrated in FIG. 3B. In one embodiment, the value of approximatedinverse rate field 221 stored in a seven bit field (thus saving a bit)ranges from 128 and 255, so if a value of less than 128 is to be stored,the value is doubled and scaling exponent 224 is increased by one (i.e.,an additional right shift or divide by two to result in the desiredvalue). Due to this scaling, the MSB of the approximated inverse rate isalways a ‘1’ and does not have to be stored.

Similarly, the fix-up adjustment value is determined, but its scaling isdifferent than that required for the approximated inverse rate (e.g.,varies between 2^6 to 2^-7 in FIG. 3B) with this relative differenceeasily determined based on the quantum and the approximated inverserate. Thus, scaling exponent 224 is also used in one embodiment to alsoscale fix-up adjustment value 222 to minimizing storage requirements.

When computing the interval, the approximated inverse rate is multipliedby the quantum, giving this product an offset related to the bitposition of the MSB of the quantum. If the exponent used with theapproximated inverse rate is treated as a number of bits to shift theapproximated inverse rate, then the exponent (right shift) used with theinterval fix-up value can be computed by the size in bits of themantissa of the approximated inverse rate minus the bit position of theMSB of the quantum (rounded up to a power of two) used with this rateencoding, plus scaling exponent 224. Note, if this result exceeds athreshold, then it is limited to allow some non-zero fix-up adjustmentvalue to bound the fix-up adjustment value to deliver the desired degreeof precision when the interval is close to one.

In one embodiment, a second scaling exponent is used such thatapproximated inverse rate 221 and fix-up adjustment value 222 are scaledindependently. In one embodiment, approximated inverse rate 221, fix-upadjustment value 222, and/or quantum 223 are floating point numbers. Inone embodiment, the exponent value used for fix-up adjustment value 222is offset by a fixed constant from scaling exponent 224 used forapproximated inverse rate 221 (or vice versa), but with a floor value(e.g., it can be no smaller than the number of bits in fix-up adjustmentvalue 222 minus one).

FIG. 3A is a flow diagram illustrating a process for determining anapproximated inverse rate, a fix-up adjustment value, and a quantumvalue used in one embodiment. Processing begins with process block 300,and proceeds to process block 302, wherein the quantum to use isdetermined based on the width of the quantum field provided. In processblock 304, the desired number of slots per byte is determined. Inprocess block 306, the stored decimal value of the approximated inverserate is determined such that it approximates the desired slots per bytein the number of storage bits provided for the approximated inverserate. In process block 308, the decimal value of the approximatedinverse rate is divided by two raised to the number of storage bits andscaled to get the approximated inverse rate. In process block 310, theinterval is determined by dividing the quantum used by the number ofbytes per slot and then by the desired rate. In process block 312, theideal fix-up adjustment value is determined by subtracting from theinterval the product of the quantum used by the actual value representedby the approximated inverse rate. In process block 314, the storeddecimal value of the fix-up adjustment value is determined to be storedin the allocated storage field; and in process block 316, the scalingexponent to use is determined. Processing is complete as indicated byprocess block 318.

Note, there are an unlimited number of methods of determining theapproximated inverse rate used in a particular embodiment, and themethod used might vary for different items within the same scheduler. Inone embodiment, for example, extra error is purposefully included in theinverse rate and/or quantum value, with, the fix-up adjustment valuecompensating for such induced error once each quantum. Also, in oneembodiment, a fix-up adjustment value is applied two or more times eachquantum, and in one embodiment, the multiple fix-up values applied for aquantum might be different values. In one embodiment, the fix-upadjustment value is applied once each quantum with a second fix-upadjustment value being applied every nth quantum to provide additionalaccuracy In one embodiment, there may be more than two levels of fix-upadjustments applied. These examples illustrate just a few of anunlimited number of such possible variations used in one embodiment inkeeping within the scope and spirit of the invention.

FIG. 3B illustrates an exemplary resultant set of approximated inverserates, fix-up adjustment values, and a quantum values for various targetrates used in conjunction with one embodiment. As shown, a 10 KB MTU and8192 slots are used.

FIG. 4A is a block diagram of a system to perform the rate computationsand/or determination of approximated inverse rates, fix-up adjustmentvalues, and a quantum values used in one embodiment. In one embodiment,system or component 400 performs one or more processes corresponding toone of the flow diagrams illustrated or otherwise described herein.

In one embodiment, component 400 includes a processing element 401,memory 402, storage devices 403, and an interface 404 for sending andreceiving packets, items, and/or other information, which are typicallycoupled via one or more communications mechanisms 409 (shown as a busfor illustrative purposes.) Various embodiments of component 400 mayinclude more or less elements. The operation of component 400 istypically controlled by processing element 401 using memory 402 andstorage devices 403 to perform one or more tasks or processes. Memory402 is one type of computer-readable medium, and typically comprisesrandom access memory (RAM), read only memory (ROM), flash memory,integrated circuits, and/or other memory components. Memory 402typically stores computer-executable instructions to be executed byprocessing element 401 and/or data which is manipulated by processingelement 401 for implementing functionality in accordance with anembodiment. Storage devices 403 are another type of computer-readablemedium, and typically comprise solid state storage media, disk drives,diskettes, networked services, tape drives, and other storage devices.Storage devices 403 typically store computer-executable instructions tobe executed by processing element 401 and/or data which is manipulatedby processing element 401 for implementing functionality in accordancewith an embodiment.

FIG. 4B is a block diagram of a system to perform the rate computationsused in one embodiment. As shown, packets 447 are received and processedby packet processor 448 to result in packets 449. The processing ofthese packets 447 is performed in response to scheduler 443, whichtypically includes its own control logic and memory. In one embodiment,rate computation circuitry (and/or processing element) 442 stores values(e.g., those illustrated in FIG. 1B or 2) in rate computation memory441. One skilled in the art would readily know how to implement inhardware the rate computations described herein, and especially thoseillustrated in FIGS. 1A and 1D, whether using a processor or typicallycombinatorial logic.

In view of the many possible embodiments to which the principles of ourinvention may be applied, it will be appreciated that the embodimentsand aspects thereof described herein with respect to thedrawings/figures are only illustrative and should not be taken aslimiting the scope of the invention. For example and as would beapparent to one skilled in the art, many of the process block operationscan be re-ordered to be performed before, after, or substantiallyconcurrent with other operations. Also, many different forms of datastructures could be used in various embodiments. The invention asdescribed herein contemplates all such embodiments as may come withinthe scope of the following claims and equivalents thereof.

1. A method for use in rate controlling an activity, the methodcomprising: identifying an approximated inverse rate of a desired rate,a fix-up adjustment value, and a quantum; maintaining an activitymeasurement value based on a measure of activity; maintaining a ratecontrol value based on the activity measurement value and theapproximated inverse rate; applying the fix-up adjustment value onceeach said quantum to the rate control value to maintain rate accuracy ofthe activity; wherein the fix-up adjustment value is a predeterminedvalue for correcting a deviation from the desired rate based on a lackof precision error induced by said use of the approximated inverse ratein maintaining the rate control value; wherein the activity includessending packets of a stream of packets; and wherein the rate controlvalue is a scheduling value used for determining the relative orderingor timing of a next one or more packets of the stream of packets.
 2. Themethod of claim 1, wherein the activity measurement value is a number ofbytes or packets sent.
 3. The method of claim 1, wherein said applyingthe fix-up adjustment value once each quantum to the rate control valueincludes dithering the rate control value to either round-up or not toround-up the rate control value based on a random number.
 4. A storagedevice storing computer-executable instructions to perform steps forrate controlling an activity, said steps comprising: identifying anapproximated inverse rate, a fix-up adjustment value, and a quantum;maintaining an activity measurement value based on a measure ofactivity; maintaining a rate control value based on the activitymeasurement value and the approximated inverse rate; applying the fix-upadjustment value once each said quantum to the rate control value tomaintain rate accuracy of the activity; wherein the fix-up adjustmentvalue is a predetermined value for correcting a deviation from thedesired rate based on a lack of precision error induced by said use ofthe approximated inverse rate in maintaining the rate control value;wherein the activity includes sending packets of a stream of packets;and wherein the rate control value is a scheduling value used fordetermining the relative ordering or timing of a next one or morepackets of the stream of packets.
 5. The storage device of claim 4,wherein the activity measurement value is a number of bytes or packetssent.
 6. The storage device of claim 4, wherein said applying the fix-upadjustment value once each quantum to the rate control value includesdithering the rate control value to either round-up or not to round-upthe rate control value based on a random number.
 7. An apparatus for usein rate controlling an activity, the apparatus comprising: means foridentifying an approximated inverse rate, a fix-up adjustment value, anda quantum; means for maintaining an activity measurement value based ona measure of activity; means for maintaining a rate control value basedon the activity measurement value and the approximated inverse rate;means for applying the fix-up adjustment value once each said quantum tothe rate control value to maintain rate accuracy of the activity;wherein the fix-up adjustment value is a predetermined value forcorrecting a deviation from the desired rate based on a lack ofprecision error induced by said use of the approximated inverse rate inmaintaining the rate control value; wherein the activity includessending packets of a stream of packets; and wherein the rate controlvalue is a scheduling value used for determining the relative orderingor timing of a next one or more packets of the stream of packets.
 8. Theapparatus of claim 7, wherein the activity measurement value is a numberof bytes or packets sent.
 9. The apparatus of claim 7, wherein saidmeans for applying the fix-up adjustment value once each quantum to therate control value includes: means for dithering the rate control valueto either round-up or not to round-up the rate control value based on arandom number.
 10. A method for use in scheduling packets, the methodcomprising: identifying in a current slot a scheduling itemcorresponding to a packet; identifying an approximated inverse rate of adesired rate, a fix-up adjustment value, and a quantum valuecorresponding to the scheduling item; identifying a last adjusted slotfor the scheduling item; adjusting a bytes sent value based on a numberof bytes of the packet to identify a new bytes sent value; and inresponse to identifying that the bytes sent value is greater than orequal to a quantum value corresponding to the scheduling item: (a)identifying a new last adjusted slot for the scheduling item, saididentifying the new last adjusted slot including summing a product ofthe approximated inverse rate and the quantum value, the fix-upadjustment value, and the last adjusted slot; and (b) determining a nextslot for the scheduling item, said determining the next slot includingadding the product of the approximated inverse rate and the new bytessent value to the new last adjusted slot; wherein the fix-up adjustmentvalue is a predetermined value for correcting a deviation from thedesired rate based on a lack of precision error induced by said use ofthe approximated inverse rate in said scheduling of packets.
 11. Themethod of claim 10, wherein said identifying the last adjusted slot forthe scheduling item includes subtracting the product of the approximatedinverse rate and the bytes sent value from the current slot.
 12. Themethod of claim 10, wherein the fix-up adjustment value is determinedbased on the error induced by the approximated inverse rate during aquantum corresponding to the scheduling item.
 13. The method of claim10, comprising: in response to identifying that the bytes sent value isless than a quantum value corresponding to the scheduling item,determining the next slot including adding the product of theapproximated inverse rate and the new bytes sent value to the lastadjusted slot.
 14. The method of claim 10, wherein said identifying thenew last adjusted slot for the scheduling item includes dithering thenew last adjusted slot to either round-up or not to round-up the newlast adjusted slot based on a random number.
 15. An apparatus for use inscheduling packets, the apparatus comprising: means for identifying in acurrent slot a scheduling item corresponding to a packet; means foridentifying an approximated inverse rate of a desired rate, a fix-upadjustment value, and a quantum value corresponding to the schedulingitem; means for identifying a last adjusted slot for the schedulingitem; means for adjusting a bytes sent value based on a number of bytesof the packet to identify a new bytes sent value; and means for inresponse to identifying that the bytes sent value is greater than orequal to a quantum value corresponding to the scheduling item: (a)identifying a new last adjusted slot for the scheduling item, saididentifying the new last adjusted slot including summing a product ofthe approximated inverse rate and the quantum value, the fix-upadjustment value, and the last adjusted slot; and (b) determining a nextslot for the scheduling item, said determining the next slot includingadding the product of the approximated inverse rate and the new bytessent value to the new last adjusted slot; wherein the fix-up adjustmentvalue is a predetermined value for correcting a deviation from thedesired rate based on a lack of precision error induced by said use ofthe approximated inverse rate in said scheduling of packets.
 16. Theapparatus of claim 15, wherein said means for identifying the lastadjusted slot for the scheduling item includes means for subtracting theproduct of the approximated inverse rate and the bytes sent value fromthe current slot.
 17. The apparatus of claim 15, wherein the fix-upadjustment value is determined based on the error induced by theapproximated inverse rate during a quantum corresponding to thescheduling item.
 18. The apparatus of claim 15, comprising: means for inresponse to identifying that the bytes sent value is less than a quantumvalue corresponding to the scheduling item, determining the next slot,which includes adding the product of the approximated inverse rate andthe new bytes sent value to the last adjusted slot.
 19. The apparatus ofclaim 15, comprising: means for computing the approximated inverse rateand the fix-up adjustment value.
 20. The apparatus of claim 15, whereinsaid identifying the new last adjusted slot for the scheduling itemincludes dithering the new last adjusted slot to either round-up or notto round-up the new last adjusted slot based on a random number.
 21. Astorage device storing computer-executable instructions to perform stepsfor use in scheduling packets, said steps comprising: identifying in acurrent slot a scheduling item corresponding to a packet; identifying anapproximated inverse rate of a desired rate, a fix-up adjustment value,and a quantum value corresponding to the scheduling item; identifying alast adjusted slot for the scheduling item; adjusting a bytes sent valuebased on a number of bytes of the packet to identify a new bytes sentvalue; and in response to identifying that the bytes sent value isgreater than or equal to a quantum value corresponding to the schedulingitem: (a) identifying a new last adjusted slot for the scheduling item,said identifying the new last adjusted slot including summing a productof the approximated inverse rate and the quantum value, the fix-upadjustment value, and the last adjusted slot; and (b) determining a nextslot for the scheduling item, said determining the next slot includingadding the product of the approximated inverse rate and the new bytessent value to the new last adjusted slot; wherein the fix-up adjustmentvalue is a predetermined value for correcting a deviation from thedesired rate based on a lack of precision error induced by said use ofthe approximated inverse rate in said scheduling of packets.
 22. Thestorage device of claim 21, wherein said identifying the last adjustedslot for the scheduling item includes subtracting the product of theapproximated inverse rate and the bytes sent value from the currentslot.
 23. The storage device of claim 21, wherein the fix-up adjustmentvalue is determined based on the error induced by the approximatedinverse rate during a quantum corresponding to the scheduling item. 24.The storage device claim 21, comprising: in response to identifying thatthe bytes sent value is less than a quantum value corresponding to thescheduling item, determining the next slot including adding the productof the approximated inverse rate and the new bytes sent value to thelast adjusted slot.
 25. The storage device claim 21, wherein saididentifying the new last adjusted slot for the scheduling item includesdithering the new last adjusted slot to either round-up or not toround-up the new last adjusted slot based on a random number.